Distribution-balanced stratified cross-validation for accuracy estimation

Cross-validation has often been applied in machine learning research for estimating the accuracies of clas-siiers. In this work, we propose an extension to this method, called distribution-balanced stratiied cross-validation (DBSCV), which improves the estimation quality by providing balanced intraclass distributions when partitioning a data set into multiple folds. We have tested DBSC V on nine real-world and three artiicial domains using the C 4:5 decision trees classiier. The results show that DBSC V performs better (has smaller biases) than the regular stratiied cross-validation in most cases, especially when the number of folds is small. The analysis and experiments based on three artiicial data sets also reveal that DBSC V is particularly eeective when multiple intraclass clusters existi n a data set.